Combining localized fusion and dynamic selection for high-performance SVM

To resolve class-ambiguity in real world problems, we previously presented two different ensemble approaches with support vector machines (SVMs): multiple decision templates (MuDTs) and dynamic ordering of one-vs.-all SVMs (DO-SVMs). MuDTs is a classifier fusion method, which models intra-class variations as subclass templates. On the other hand, DO-SVMs is an ensemble method that dynamically selects proper SVMs to classify an input sample based on its class probability. In this paper, we newly propose a hybrid scheme of those two approaches to utilize their complementary properties. The localized fusion approach of MuDTs increases variance of the classification models while the dynamic selection scheme of DO-SVMs reduces the unbiased-variance, which causes incorrect prediction. We show the complementary properties of MuDTs and DO-SVMs with several benchmark datasets and verify the performance of the proposed method. We also test how much our method could increase its baseline accuracy by comparing with other combinatorial ensemble approaches. Expert systems on various domains have been employing clas-sifier ensemble approaches to deal with the complex prediction and classification problems of real-world applications (García-A classifier ensemble incorporates multiple classifiers to achieve highly reliable and accurate performance in classification (Jain, 2000). Fusion and selection are two main streams in ensembling, where the purpose of fusion is to combine diverse classifiers (Brown, Wyatt, Harris, & Yao, 2005; Windeatt, 2004) while the selection's is to choose a competent (locally accurate) classifier for a test sample (see Fig. 1). The selection approach is often further categorized into static or dynamic methods according to whether the selection regions on a sample space are specified during the training phase or operation phase, respectively (Kuncheva, 2002). In general, a fusion approach could be less adaptive to an incoming sample than a selection method since the fusion uses the same combination of base classifiers for all the incoming samples. On the other hand, the selection approach might produce biased results by relying on a chosen classifier. To address those weaknesses, recent studies have introduced combinatorial ensemble approaches that select competent classifiers and combine their Ensemble approaches also have been applied to combine multiple binary classifiers like support vector machines (SVMs) García-In our previous work, we presented two different ensemble methods with SVMs: multiple decision templates (MuDTs) Min, Hong, & Cho, 2010 and dynamic ordering of one-vs.-all SVMs (DO-SVMs) Hong, Min, Cho, & Cho, 2008. MuDTs is a classifier fusion method that models intra-class variations of a given …